school system), and the evaluators. When the sponsor is a federal agency, there are three control points within the agency:  the evaluation monitor, the contracts office, and the manager of the program being evaluated. The complexities created by these multiple organizational relationships create constraints for any study, and those constraints have been given little attention. Our own limited findings related to such issues are reported in the next three chapters; those findings make it clear that the evaluation process must be better understood if it is to yield good results.
The National Institute of Education should encourage work in the noted areas of methodology and process as part of its evaluation research program. Furthermore,, with rare exceptions, when a specific methodological question must be addressed in a given time frame or the process of a specific evaluation is to be studied, all such research should be carried out through a competitive grants program that specify the areas of interest but not the approach to be taken.
NOTES
Success here is defined in terms of the objectives of the program.  It is quite possible that a program successful with respect to its own objectives may be educationally undesirable. For example, perhaps more time was spent on a targeted skill and so some other important skill was neglected and hence less developed than it would have been in the absence of the program. To gauge the overall educational contribution of a program, it is necessary to assess such negative as well as the positive effects. A good deal of knowledge that can be applied to program improvement may, in fact, be gained through documenting program variations and their effects. A panel of the National Research Council's Committee on Child Development Research and Public Policy is currently reviewing outcome measurement in early childhood demonstration programs. Given that local program variation is encouraged by many early